Matrix Completion: Fundamental Limits and Efficient Algorithms a Dissertation Submitted to the Department of Electrical Engineering and the Committee on Graduate Studies of Stanford University in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy

iv Preface Low-rank models provide low-dimensional representations capturing the important aspects of data naturally described in matrix form. Examples range from users' preferences on movies to similarities between pairs of items. Naturally, the low-rank representations serve as a tool for data compression and efficient computation. But more importantly, they are used in data analysis to learn hidden structures of the data, which is at the heart of machine learning and data mining. Applications include latent semantic analysis, factor analysis, and clustering high-dimensional data. While singular value decomposition provides an efficient way to construct a low-rank model of a fully observed data matrix, finding a low-rank model becomes challenging when we only have a partial knowledge about the matrix. When we observe a subset of the entries, we show that the singular value decomposition is sub-optimal and can be significantly improved upon. In this work, we investigate the possibility of learning a low-rank model from a partial observation of a data matrix. We develop a novel and efficient algorithm that finds a near-optimal solution. Further, we prove that the proposed algorithm achieves performance close to the fundamental limit under a number of noise scenarios. This provides a solution to many practical problems including collaborative filtering and positioning. v vi Acknowledgement During my years at Stanford, I was fortunate to have Andrea Montanari as my advisor. I learned everything I know about research from Andrea. Since the beginning of my PhD, I've continuously benefited from long meetings with Andrea, who was always ready to write down equations with me and to listen to the numerous talk rehearsals. I am grateful for his patience and support. Further, I learned from Andrea the joy of research. Everytime we discussed new ideas, I could see his enthusiasm and excitement. I am truly thankful to Andrea for sharing his passion and for showing me that research is fun, and I am proud and honored to be his student. I also enjoyed the advice and guidance from a number of other mentors, who will have a lasting influence on my research direction and my life. Tom Cover has been a constant source of inspiration from the beginning of my graduate study. I am really fortunate to have him as a teacher and a mentor. I would like to thank Balaji Prabhakar for his support and encouragement. His door was always open when I needed help. I have …